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Objective It was proposed that the ability to recognize facial emotions is closely related to complex neurocognitive processes and/or 
skills related to theory of mind (ToM). This study examines whether ToM skills mediate the relationship between higher neurocognitive 
functions, such as reasoning ability, and facial emotion recognition. 

Methods A total of 200 healthy subjects (101 males, 99 females) were recruited. Facial emotion recognition was measured through the 
use of 64 facial emotional stimuli that were selected from photographs from the Korean Facial Expressions of Emotion (KOFEE). Partic- 
ipants were requested to complete the Theory of Mind Picture Stories task and Standard Progressive Matrices (SPM). 

Results Multiple regression analysis showed that the SPM score (t=3.19, p=0.002, (3=0.22) and the overall ToM score (t=2.56, p=0.011, 
|3=0.18) were primarily associated with a total hit rate (%) of the emotion recognition task. Hierarchical regression analysis through a 
three-step mediation model showed that ToM may partially mediate the relationship between SPM and performance on facial emotion 
recognition. 

Conclusion These findings imply that higher neurocognitive functioning, inclusive of reasoning, may not only directly contribute to- 
wards facial emotion recognition but also influence ToM, which in turn, influences facial emotion recognition. These findings are par- 
ticularly true for healthy young people. Psychiatry Investig 2014;11(2):105-111 
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INTRODUCTION 

Facial emotion recognition abilities are crucial for the de- 
velopment of successful relationships and healthy psychologi- 
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cal functioning.' ' The most widely used measure of facial 
emotion recognition is the emotion -labeling test, which re- 
quires subjects to choose an emotional category that best de- 
scribes the emotional states of the person in the photograph 
shown. This test requires complex neurocognitive processes. 
According to previous studies on healthy controls'*'^ and clini- 
cal patients'' '' * with brain disorders such as schizophrenia, 
these neurocognitive processes may include the visual scan- 
ning of facial features and vigilance, which facilitates readiness 
to respond to relevant stimuU;'' working memory, which facili- 
tates the retention of information on line long enough for the 
test-taker to compare the facial photograph with the various 
labels for emotional categories;^'* semantic memory, used for 
encoding and retrieving emotional labels;' concept formation 
and cognitive flexibility, which are executive functions re- 
quired for choosing the appropriate emotion label and chang- 
ing that label during each emotion recognition trial;^ and the 
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reasoning function of abstraction so as to recognize and dis- 
tinguish between different facial emotional stimuli.'^ 

In addition to neurocognition, theory of mind-the ability 
to make inferences about the mental states of others'°-may be 
relevant to facial emotion recognition. Individuals do not in- 
terpret the emotional expressions of others on the basis of sur- 
face meaning alone, but instead incorporate intentions, knowl- 
edge, and beliefs. The integration of all these factors results in 
unique interpretations of events. According to Bryson et al's 
proposal regarding the contribution of ToM towards facial 
emotion recognition, in order to label individuals' internal ex- 
periences on the basis of behavioral signs occurring as a result 
of their internal states, subjects should have created a concept 
of internal space for themselves and others, which is a core 
component of ToM. Thus, the facial emotion recognition test 
should also assess one's ability to acknowledge the existence 
of others' internal states. Recently, five core constructs for so- 
cial and emotional behavior were proposed to have distinct- 
tive and interactive functions:" 1) the acquisition of social- 
affective values and responses, 2) recognition of social- 
affective stimuli, 3) embodied simulation or low-level mental 
state inference, 4) high-level mental state inference, and 5) 
context-sensitive regulation, which might operate indepen- 
dently or in various combinations. According to this model, to 
understand the meaning of facial emotions, not only the ac- 
quisition of the social-affective values, but also low-level simu- 
lation (low-level ToM) and higher-level ToM skills were need- 
ed. According to Goldman's (2005) view of the contribution of 
low-level simulation, the visual representation of a target's fa- 
cial expression would induce an emotional experience within 
the subject via a simulation process; the subject can then clas- 
sify his or her current emotional state and attribute it to the 
target.'^ For the accurate attribution of the subject's emotional 
state to the target, the subject should differentiate himself or 
herself from others, which is a core component of ToM. Re- 
garding the possible contribution of higher-level ToM skills," 
the meaning of a facial expression is determined by the con- 
text in which the former is displayed. For example, a smiling 
face may be an expression of unambiguous happiness or sar- 
casm, depending on the context. To accurately recognize a fa- 
cial expression within the various contexts, a third-person 
perspective is needed, to integrate the context with the facial 
expression. Taking a third-person perspective is also a core 
component of ToM. In fact, though relatively few, some stud- 
ies found that ToM skills were associated with facial emotion 
recognition among healthy subjects'^ and clinical patients with 
schizophrenia," as well as autistic disorder.'^ 

In addition, several studies have reported relations between 
neurocognition and ToM. More specifically, the important 
neurocognitive processes supporting ToM were found to be 



the executive functions of attention flexibility, inhibition of 
default information,''' and the updating of information in 
working memory"" among healthy adults"" and children'^ as 
well as clinical patients, including those suffering from trau- 
matic brain injury'* and schizophrenia."'^" 

Therefore, ToM may play a mediating role in the relation- 
ship between higher neurocognitive functions, such as reason- 
ing ability, and facial emotion recognition. This potential role 
of ToM is important to elucidate the pathogenetic mechanism 
of facial emotion recognition deficits in clinical populations, 
which showing the poor performances on the verbal and/or 
pictorial stories tasks of ToM skills. On the basis of previous 
findings, we hypothesized that reasoning ability and ToM were 
independent predictors of facial emotion recognition. More 
importantly, ToM may mediate the relationship between rea- 
soning ability and facial emotion recognition in healthy peo- 
ple. We used facial photographs expressing basic emotions 
without any kind of contextual information, which may result 
in an ambiguous context for tapping the contribution of ToM 
skills towards performance on facial emotion recognition. To 
definitively conclude that ToM mediates the relationship be- 
tween reasoning abiUty and facial emotion recognition, three 
conditions must be met: 1) the potential mediator (i.e., ToM) 
must be significantly associated with reasoning ability, 2) the 
facial emotion recognition variable must also be significantly 
associated with reasoning ability, and 3) the potential mediator 
(i.e., ToM) must still be associated with facial emotion recog- 
nition when included in a multiple regression model with rea- 
soning ability (Figure 1). It is expected that the effect of rea- 
soning ability will decrease in potency since the mediator (i.e., 
ToM) will explain a portion of the effect. 
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Figure 1. ToM mediates the relationship between reasoning abili- 
ty and facial emotion recognition, three conditions must be met: 
equation 1) the potential mediator (i.e., ToM) must be significantly 
associated with reasoning ability, equation 2) the facial emotion 
recognition variable must also be significantly associated with 
reasoning ability, and equation 3) the potential mediator (i.e., 
ToM) must still be associated with facial emotion recognition 
when included in a multiple regression model with reasoning abil- 
ity. *p<0.05, **p<0.01,***p<0.001. 
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METHODS 
Subjects 

A total of 200 healthy subjects (101 males, 99 females) were 
recruited via an Internet job advertisement from August 201 1 
to July 2012. All subjects met the inclusion criterion of being 
between 15 and 25 years old. We selected this age group, as we 
will be conducting a similar study on peoples peak age during 
the onset of psychiatric illness. Individuals with current or past 
psychiatric or neurological illness history, or traumatic brain 
injury, were excluded on the basis of a preliminary interview 
that was conducted telephonicaUy Written informed consent 
was obtained from all participants after the procedure had 
been fiiUy explained to them. The study was reviewed and ap- 
proved by the Institutional Review Board of the Severance Me- 
ntal Health Hospital. For participants below the age of 18 
years, we also obtained informed consent from their parents. 
All participants' data were used for analysis. 

Participants' mean age and years of education were 20.5 
(2.0) and 13.4 (1.4), respectively 

Facial emotion recognition task 

The 64 Korean facial stimuli were selected from photogra- 
phs from the Korean Facial Expressions of Emotion (KOFEE) 
database (Figure 2).^' The stimuli were standardized in our 
laboratory, which is commonly utilized for research on emo- 
tion recognition in Korea. The KOFEE comprises photo- 
graphs of seven universal facial expressions (happiness, dis- 
gust, anger, sadness, surprise, fear, and contempt) and neutral 
faces, which Korean models posed for All facial expressions 
included in the KOFEE were coded through the use of Ek- 
man and Friesen's Facial Action Coding System (FACS), 
which implies that each expression is a direct analog of ex- 
pressions that actually occur in real life when people express 
their emotions faciaUy.^^ Sbcty-four photographs (8 photo- 
graphs of each emotion and 8 neutral faces) showed accept- 
able inter-rater agreement rate (>70%) in our previous 



standardization study^' (n=105, Korean college students). 
The selected photographs were adjusted by using the Phanta- 
Morph 5.0 (Abrosoft, USA). The selected 64 photographs in 
the facial emotion recognition task were accompanied by a re- 
sponse sheet placed below the facial photograph. Subjects were 
asked to choose an emotion category that best describes the 
emotional states of the person in the photographs, based on 
the facial expression at their own pace. A forced choice format 
was provided for subjects to choose among the categories of 
"happiness," "disgust," "anger," "sadness," "surprise," "fear^ "con- 
tempt," and "neutral" for the facial stimuli that were being pre- 
sented. The construction of the task such that the emotional 
face and labels appeared on the same computer monitor until 
subjects made a choice did not necessitate more encoding and 
retrieval ability by semantic memory.^ In the present study, the 
total hit rate (%) was calculated as the percentage of correct re- 
sponses for 64 facial photographs. The facial stimuli and re- 
sponse sheet were presented and the responses were recorded 
using by the Neuroguide (Applied Neuroscience, USA), E- 
Prime 2.0 (Psychology Software, USA), and Inquisit 3.0 (MMi- 
second software, USA). 

Analogical reasoning task 

The Standard Progressive Matrices (SPM)^' test was devel- 
oped to measure the function of formal reasoning by analogy, 
which is closely associated with the executive fiinctions of the 
prefrontal cortex.^"* Formal reasoning requires the integration 
of relations as well as the ability to infer rules and form high- 
level abstractions, while other reasoning tests require the use 
of real-world knowledge as well as inhibition of salient dis- 
tracters.^^ The SPM test consists of 60 non-colored diagram- 
matic puzzles, each with a missing part, which subjects were 
asked to identify from several options. This test, which has 
been widely used in Korea,^*" has been shown to have high reli- 
ability and validity among various cultural groups, regardless 
of the amount of time provided.^^ The SPM score was calculat- 
ed as the sum of correct answers. 




Figure 2. Korean facial stimuli were selected from photographs from the Korean Facial Expressions of Emotion (KOFEE). 
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Theory of mind task 

The Theory of Mind Picture Stories task^* was developed by 
Briine to measure the higher- order ToM ability. In this task, 
ToM is assessed using a series of novels of six cartoon picture 
stories; each story was made up of four picture cards. Subjects 
were requested to turn the cards that were presented face 
down in the same order (4-1-2-3) and rearrange them in a 
logical sequence of events. A sequencing score was allocated 
for each picture story. Two points were given if the first and 
last cards were in the correct sequence, and one point was giv- 
en if the second and third cards were in the correct sequence 
(0-36 points). The questionnaire score consisted of scores ob- 

Table 1 . Descriptive statistics for study variables (N=200) 



Variable 


Mean 


SD 


Range 


KOFEE* 


84.7 


8.4 


56.3-98.4 


SPMt 


51.7 


5.4 


15.0-60.0 


ToM* 


55.0 


3.9 


43.0-59.0 



*totaI hit rate (%) of facial emotion recognition task score of the 
Korean Facial Expressions of Emotion (KOFEE),^^ ^number of 
correct answers of Standard Progressive Matrices,^*" tsum of se- 
quencing and questionnaire score of Theory of Mind Picture Sto- 
ries task'' 

Table 2. IVIultiple regression analysis of domain variables and 
KOFEE* (N=200) 



Predictor Outcome 



p-value 



Sex 


KOFEE 


0.10 


1.42 


0.157 


Age 


KOFEE 


-0.02 


-0.31 


0.756 


SPMt 


KOFEE 


0.22 


3.19 


0.002 


ToM* 


KOFEE 


0.18 


2.56 


0.011 



*totaI hit rate (%) of facial emotion recognition task score of the 
Korean Facial Expressions of Emotion (KOFEE),^^ ^number of 
correct answers of Standard Progressive Matrices,^' tsum of se- 
quencing and questionnaire score of Theory of Mind Picture Sto- 
ries task" b: unstandardized beta coefficient, SE: standard error, p: 
standard beta coefficient, t: t-value 



tained from the 23 points pertaining to the mental states of the 
cartoon characters that were given (e.g., "What does the blond 
haired person believe is in the box," "What does the shopgirl 
now think the boys intended to do?").This task yielded a se- 
quencing score (0-36) on logical reasoning and a question- 
naire score (0-23) on the comprehension of first-/second-and 
third-order belief as well as false belief reality reciprocity de- 
ception, and cheating detection. The overall ToM score was 
calculated as the sum of the sequencing and questiormaire 
scores (0-59 points) (for further details^'). 

Procedures 

Each subject, when performing the Theory of Mind Picture 
Stories task, was assessed by a psychiatrist (LSB) and a psy- 
chologist (LMK, KSJ), followed by the completion of comput- 
erized SPM and the facial emotion recognition task in a con- 
trolled laboratory. All tasks took approximately 50 minutes to 
complete. 

Statistical analysis 

In the present study, all scores were screened for outliers, us- 
ing Tukey s hinges and, if possible, outliers were included after 
a winsorizing procedure.'" Following data screening, the skew- 
ness and kurtosis of all rated scores were in an acceptable 
range (<1.0). 

Multiple regression was used to examine the hypothesized 
variables' ability to predict the overall hit rate (%) of the emo- 
tion recognition task. The enter method was utilized in the 
multiple regression model. The mediation procedure suggest- 
ed by Baron and Kenny'' was employed to examine whether 
the overall ToM score acts as a mediator in the relationship 
between performance on the facial emotion recognition task 
and SPM scores. Three sequential regressions were tested: 1) 
According to the test that analogical reasoning ability explains 
the variance in ToM, scores on the Theory of Mind Picture 



Table 3. Theory of mind: test for its mediation of SPIVIt to KOFEE*: equation 1, 2 (bivariate regressions) and equation 3 (multiple regres- 
sion) (N=200) 



IV 



Equation one 
DVToMsp t-stat(p) 



Equation two 
DV KOFEE p t-stat(p) R' 



IV 



Equation three 
DV KOFEE p lvalue (p) 



df 



SPM 



0.20 



2.94** 0.04 
(0.004) 



0.25 



3.67*** 0.06 Block 1: 
(<0.001) SPM 

Block 2: 

1. SPM 

2. ToM 



0.25 3.67*** 13.48 0-06 198 

(<0.001) (<0.001) 

0.22 3.14** 
(0.002) 

0-17 2.46* 9.94 0.09 197 

(0.015) (<0.001) 



*p<0.05, **p<0.01 , ***p<0.001 , tnumber of correct answers of Standard Progressive Matrices,^'' ttotal hit rate (%) of facial emotion recogni- 
tion task score of the Korean Facial Expressions of Emotion (KOFEE),^ §sum of sequencing and questionnaire score of Theory of Mind Pic- 
ture Stories task.'' SPM: The Standard Progressive Matrices, DV: dependent variable 
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Stories task were found to regress on SPM scores (equation 1 
in Figure 1). 2) Facial emotion recognition was regressed on 
analogical reasoning ability, testing whether SPM scores ex- 
plain the variance in facial emotion recognition (equation 2 
in Figure 1). 3) When analogical reasoning ability was in- 
cluded in the multiple regression model, ToM still explained 
the variance in facial emotion recognition through a test sho- 
wing that facial emotion recognition was regressed on scores 
for both ToM and SPM (equations 2 and 3 in Figure 1). We 
also conducted Sobels test'^ by using unstandardized regres- 
sion coethcients and standard errors in order to evaluate the 
significance of the indirect effect when the mediator is added 
to the model. MulticoUinearity was not a concern for any of 
the variables that were included in the analysis (highest vari- 
ance inflation factor=1.07). 

RESULTS 

Performance on the facial emotion recognition task, 
analogical reasoning, and ToM task 

Scores on each task are summarized in Table 1. 

Predictors of facial emotion recognition 

Multiple linear regression analysis was performed where the 
total hit rate (%) on the emotion recognition task was entered 
as an outcome variable, and age, sex, SPM score, and the over- 
all ToM score were entered as predictor variables. The overall 
model was significant [R'=0.10, F (4,195)=5.57, p<0.001]. 
SPM scores (t=3.19, p=0.002, (3=0.22) and overall ToM scores 
(t=2.56, p=0.011, (3=0.18) were both significant predictors. The 
main results are summarized in Table 2. 

Mediation analysis 

Baron and Kenny's'' approach was employed to determine 
whether ToM mediates the significant relationship between 
SPM score and performance on the facial emotion recogni- 
tion task (Table 3). We also conducted a three-step hierarchi- 
cal regression analysis to determine whether overall perfor- 
mance on the ToM task mediates the relationship between 
SPM score and performance on the facial emotion recognition 
task. In the first step (equation 1), the linear regression analy- 
sis revealed that the SPM score was significantly related to 
the overall ToM score (t=2.94, df=198, p=0.004, (3=0.20). In 
the second step (equation 2), it was also found that the SPM 
score significantiy predicted the total hit rate on the emotion 
recognition task (t=3.67, df=198, p<0.001, (3=0.25). When 
the overall ToM score was added in the third step (equation 
3), it was significantly associated with the total hit rate on the 
facial emotion recognition task (t=2.46, df=197, p=0.015, 
(3=0.17), and the SPM score became a weaker predictor than it 



was in the second step (t=3.14, df=197, p=0.002, (3=0.22). In 
addition, Sobels test revealed a significant indirect effect of the 
overall SPM score on the total hit rate on the facial emotion 
recognition task when the overall ToM score was added to 
the model (Z=2.14, p=0.034). Thus, ToM may partially medi- 
ate the relationship between SPM and performance on the fa- 
cial emotion recognition. 

DISCUSSION 

To the best of our knowledge, this is the first study to inves- 
tigate the role of ToM as a mediator of the relationship be- 
tween reasoning ability and facial emotion recognition on 
with a large sample (n=200) of healthy subjects. The main 
finding from our study is that reasoning ability and ToM skills 
are independent predictors of facial emotion recognition. 
More importantiy, ToM was found to partially mediate the re- 
lationship between reasoning ability and facial emotion recog- 
nition. 

Analogical reasoning (as measured by SPM) was indepen- 
dentiy related to facial emotion recognition. This association is 
essentially consistent with previous findings that the executive 
function of abstraction and verbal fluency in healthy controls' 
correlates with facial emotion recognition ability. Analogical 
reasoning indexed by SPM requires the integration of rela- 
tions, as well as the ability to infer rules and form abstractions 
on the basis of non-social information, which is in turn closely 
associated with the executive function of the prefrontal cor- 
tex.^''' A possible explanation is that when subjects recognize 
others' facial emotion photographs at their own pace, analogi- 
cal reasoning ability, which is based on the information of fig- 
ural matrices, may be needed to decode the complex visuospa- 
tial stimuU, as well as integrate and infer the emotional status 
of "posers," or models posing with different facial emotional 
expressions. 

An association between ToM skills and facial emotion rec- 
ognition was also found. This finding is explained in terms of 
the relationships between the five proposed core constructs of 
social and emotional behavior" According to this model, 
higher-level ToM skills may be involved during attempts to 
understand the meaning of a social-affective stimulus such as 
facial emotions. Alternately, and more specifically, these find- 
ings suggest the role of false-belief and perspective-taking, 
which are a part of ToM skills, as it is essential for subjects to 
know that the poser's emotional state in the photograph was 
not evoked by the subject himself or herself despite being in 
front of the poser's face. This formulation may be robust for 
faces with negative emotions such as fear, since subjects with 
poor false-belief ToM skills and no intention to harm others 
may not easily understand the poser's expression of fear 
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(thinking "Why is he/she afraid of me? I am innocent.") and 
therefore do not select "fear" (thinking "His/her expression 
should not show fear because there is no reason for him/her to 
be afraid of me"), while subjects with intact ToM skills and no 
intention to harm others would know that the poser's expres- 
sion of fear is not elicited by them (thinking "Why is he/she 
afraid? It is not my fault, because I was not front of him/her 
when the photograph was taken, although at present I am in 
front of his/her face in the photograph."), and thus choose 
"fear" in the response sheet ("His/her expression should be 
fear, although I do not know the details of the situation that 
made him/her afraid."). 

With regard to the influence of ToM skills on the relations 
between higher neuro cognition and facial emotion recogni- 
tion, ToM skills seemingly acted as a partial mediator of the 
relationship between analogical reasoning and facial emotion 
recognition. High-order neurocognitive processing of infer- 
ence and abstraction, indexed by SPM using non-social tigural 
matrices, may also be a necessary component of ToM skills, 
which include logical sequencing and comprehension of false- 
belief reciprocity, deception, and detection of cheating. In our 
study, the facial emotion recognition task may require higher- 
level neurocognitive functioning such as analogical reasoning, 
so that subjects can label the internal experience of the poser 
in the photograph, on the basis of surface information of com- 
plex visuospatial features evident in the posers facial expres- 
sion and the generated concept of the internal space of both 
the subject and the poser, which simultaneously capture the 
perspective- taking of ToM skills and reasoning ability. There- 
fore, analogical reasoning may not only directly contribute to- 
wards facial emotion recognition, but also influence ToM sk- 
ills, which in turn influence facial emotion recognition. 

A limitation of our study is that the measurement of neuro- 
cognitive fiinctioning was restricted to one higher neurocog- 
nitive fiinction, namely, analogical reasoning ability (SPM). 
There is more than one way to assess neurocognitive function- 
ing; arrange of assessments could have therefore offered a 
more comprehensive set of results. We assessed only higher- 
level ToM skills for the social domain, but not low-level mental 
state inferences, or the index of lower- level self-boundary and 
embodied simulation. In future, more comprehensive assess- 
ments of neurocognitive, as well as higher and lower-level of 
ToM skills measurement, will be required to explain the role of 
ToM skills as a mediator of the relationship between neuro- 
cognitive functions and facial emotion recognition. 

In summary, we found that reasoning ability and ToM were 
independent predictors of facial emotion recognition in heal- 
thy people. More importantly, ToM was found to partially me- 
diate the relationship between reasoning ability and facial 
emotion recognition. These findings imply that higher neuro- 



cognitive functions, including reasoning, may not only directly 
contribute towards facial emotion recognition but also influ- 
ence ToM, which in turn influences facial emotion recogni- 
tion. These findings also highlight the need for further re- 
search to explore the mechanisms underlying deficits in facial 
emotion recognition, with respect to ToM and neurocognition 
among individuals with psychiatric illnesses such as schizo- 
phrenia and those at an ultra-high risk for psychosis. 
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